Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

infosync: integrate PD HTTP client into the label manager #48738

Merged
merged 4 commits into from
Nov 23, 2023

Conversation

JmPotato
Copy link
Member

What problem does this PR solve?

Issue Number: ref #35319, tikv/pd#7300

Problem Summary:

Integrate PD HTTP client into the label manager.

What is changed and how it works?

  • Replace the etcd client inside LabelRuleManager with PD HTTP client.
  • Reuse the type definitions within the PD SDK as much as possible.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Nov 21, 2023
Copy link

codecov bot commented Nov 21, 2023

Codecov Report

Merging #48738 (fb36395) into master (9062834) will increase coverage by 1.7752%.
Report is 18 commits behind head on master.
The diff coverage is 55.2083%.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #48738        +/-   ##
================================================
+ Coverage   71.0864%   72.8617%   +1.7752%     
================================================
  Files          1365       1392        +27     
  Lines        404572     413041      +8469     
================================================
+ Hits         287596     300949     +13353     
+ Misses        97010      93148      -3862     
+ Partials      19966      18944      -1022     
Flag Coverage Δ
integration 43.7541% <40.6250%> (?)
unit 71.2905% <55.2083%> (+0.2041%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 53.9663% <ø> (ø)
parser ∅ <ø> (∅)
br 49.0092% <ø> (-3.9652%) ⬇️

@JmPotato JmPotato force-pushed the adopt_pd_http_cli_to_infosyncer branch 2 times, most recently from 6d64793 to 7a7796b Compare November 21, 2023 09:46
tikv.WithPDHTTPClient([]string{ts.Addr()}),
),
)
ts.store, err = mockstore.NewMockStore()
Copy link
Member Author

@JmPotato JmPotato Nov 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous commit made changes to this code section, but it was later determined to be unnecessary during testing. This PR reverts those changes.

0f35b6f#diff-69c54bd9e8a29f957a0cb3d46941079117eadecfa6090abfac3bd05f78ec37e1L453

@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Nov 22, 2023
@JmPotato JmPotato force-pushed the adopt_pd_http_cli_to_infosyncer branch from 7a7796b to fb36395 Compare November 22, 2023 05:40
Copy link

tiprow bot commented Nov 22, 2023

@JmPotato: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
tiprow_fast_test fb36395 link true /test tiprow_fast_test

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@JmPotato
Copy link
Member Author

/test check-dev2

Copy link

tiprow bot commented Nov 22, 2023

@JmPotato: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test tiprow_fast_test

Use /test all to run all jobs.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@JmPotato
Copy link
Member Author

/test unit-test

Copy link

tiprow bot commented Nov 22, 2023

@JmPotato: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test tiprow_fast_test

Use /test all to run all jobs.

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@JmPotato
Copy link
Member Author

PTAL

/cc @tangenta @zimulala

@ti-chi-bot ti-chi-bot bot requested review from tangenta and zimulala November 22, 2023 08:43
@@ -209,7 +209,10 @@ func GlobalInfoSyncerInit(
if err != nil {
return nil, err
}
is.labelRuleManager = initLabelRuleManager(etcdCli)
if pdHTTPCli != nil {
pdHTTPCli = pdHTTPCli.WithRespHandler(pdResponseHandler)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we put it to initLabelRuleManager? It seems we use pdHTTPCli only in initLabelRuleManager.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Later we will integrate the PD HTTP client into the Placement Rule manager below also, so I keep it initialized here.

func NewRulePatch(setRules []*Rule, deleteRules []string) *pd.LabelRulePatch {
labelRules := make([]*pd.LabelRule, 0, len(setRules))
for _, rule := range setRules {
labelRules = append(labelRules, (*pd.LabelRule)(rule))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does changing the type of setRules(line157) to pd.LabelRule also eliminate the need for conversions?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main consideration here is that TiDB has made additional extensions to the SQL layer concept for the Rule type internally. Therefore, I decided to keep the relevant implementation of the Rule type in TiDB and hide the type conversion in the lower layer from the caller. Otherwise, we would have needed to completely replace Rule with pd.LabelRule, or constantly introduce type conversions throughout. However, prioritizing code readability over a slight performance sacrifice seems like a better choice.

Copy link
Contributor

@zimulala zimulala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

ti-chi-bot bot commented Nov 23, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hawkingrei, zimulala

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added approved lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Nov 23, 2023
Copy link

ti-chi-bot bot commented Nov 23, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-11-22 02:15:56.939354433 +0000 UTC m=+370585.604580629: ☑️ agreed by hawkingrei.
  • 2023-11-23 04:09:34.432978852 +0000 UTC m=+463803.098205046: ☑️ agreed by zimulala.

@ti-chi-bot ti-chi-bot bot merged commit 7e8de2f into pingcap:master Nov 23, 2023
15 of 16 checks passed
@JmPotato JmPotato deleted the adopt_pd_http_cli_to_infosyncer branch November 23, 2023 04:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants